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| Abstract 



We study the stability of the dynamics of a network of n formal neurons interacting through 
an asymmetric matrix with independent random Gaussian elements of the type introduced by 
Raj an and Abbott (([ID]), The neurons are represented by the values of their electric potentials 
Xi,i = 1, . . . , n. Using the approach developed in a previous paper by us ([7]) we obtain sufficient 
conditions for diverging synchronized behavior and for stability. 



> 
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C\i ■ 1 Introduction 
t-h ; 

The dynamic of a system of neurons described by their electric potentials Xi(t),i = 1, . . . , n inter- 
acting linearly through a random matrix has been extensively studied in the past literature and 
received increased attention in the last times, see for example (p], [2], [5]), [3]). The first statement 
about the stability of the solution of the system 



x' = -kx + J'x, (1.1) 
was enunciated by May ([9J). Here J' is a real symmetric n x n matrix with independent 



the maximum eigenvalue of J', then the solutions of the system (|1.1|) were stable. This conjecture 



gaussian elements and EJ'- = 0, EJ? = 1/n. The conjecture was that if k > A max , where A max i s 



■ has been proved by us many years later in the paper ([7J) where the self-averaging property of the 

system have been used. In particular we introduced the random counting measure 

n 

Af n (\, t) = n-H{ Xi {t) < A} = n- 1 £ 9(\ - Xi (t)), (1.2) 

i=l 

where 6{x) is the standard Heaviside function. This function N n counts the fraction of the 
electric potentials Xi(t) which are less than a given threshold A. The self-averaging property of 
J\f n means that M n (t) — > EAf n (t) in the L 2 norm with respect to the probability measure of the 
gaussian matrix, E being the expectation with respect to the probability of all the random entries 
of the matrix J'. In (|7J) we were able to proof this property, so the random measure becomes 
asymptotically a gaussian distribution function with mean value a(t) and dispersion cr(t): 

rX -(x-a(t)) 2 /2a(t) 

ton E{J\f n (\,t)} = / dx . (1.3) 

The results of the calculations were that a(t) = e~ Kt and a(t) = e~ 2Kt Jo(wt) where Jo is the 
Bessel function of zero order. From the asymptotic behavior of Jo we get that if k > w a(t) goes 
to zero and we get a stable solution. It is a quite remarkable coincidence that this nice result 
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depends on the self-averaging property of the system, this shows the real power of such property if 
one reminds all the rigorous properties which have been possible to show in the field of Statistical 
Mechanics of disordered systems. In this paper we look for analogous results when a different 
matrix J' is considered. The elements of J' are still independent but each row of the matrix have 
mean values depending on the column index: 



prrM_„ J 1*1 n , j = L,...[jn, i , . 



Ai//n 1/2 , j = l,...[/n] 
VE/n 1/2 , j = [fn] + 1, . . . ,n. 

The first [/n] columns represent inhibitory interaction (/i/ < 0) while the other n — [/n] are 
excitatory interaction (ue > 0), thus each neuron i receives [/n] inhibitory inputs of the same 
type from the other neurons and n — [fn] excitatory inputs form the other neurons, the excitation 
and the inhibition do not depending on the particular neuron i. With this choice the matrix J' is 
asymmetric and the variances of the matrix elements of J' also follow the same choice: 



E{{JL-E{JLm = n-^= J r „ (1.5) 



crj/n, j = l,...[fn], 
o-e/ti, j = [fn] + 1,... ,n. 

Thus we look at the same property as before in the case of this new matrix which includes 
inhibitory and excitatory inputs which is nearer to realistic neural interactions. In order to un- 
derstand better the new kind of stability properties that we obtain let us introduce some more 
definitions. Let m = (mi, . . . , m n ) be the vector defined by 

m .-(<- j = l,-..[fn] (16) 



and M be the matrix with all rows equal to m. Then in the paper, following the ideas of (QlOj), 
we introduce the decomposition 

j' = J + aM, (1.7) 

in order to have all the eigenvalues included in the circle with radius one in the complex plane. 
The two theorems shown in this paper describe in detail the stability and asymptotic properties of 
the dynamic associated to the matrix J' and in particular these properties are very sensible to the 
choice of the initial conditions for the Xi(t). We give here some hint since the complete definitions 
will be given in the next section. So suppose that the initial conditions can be written in the 
following way: 

Xi(0)=Ci + Zi, (1.8) 
where are independent random variables with distributions {vi}, satisfying the conditions 

£fe} = 0, Em 2 } = af\ E{(Ci) s }<C. (1.9) 
and the initial constants depend on the neuron in the same way as the 



cj, i = l,...[fn], 
ce, otherwise, 



;i.io) 



In this situation the theorems proved in the paper establish that the contribution to the dynamic 
of the matrix J is stable if k > ex* = faj + (1 — f)o'E- Remark that since the matrix J is asymmetric 
the stability of the dynamic does not depend on the maximum eigenvalue so it is reasonable that 
the stability results for the dynamic generated by this matrix is different from the one enunciated 
above. The matrix M also contributes to the dynamic and, due to its particular form, gives some 
unexpected result, namely if c/ 7^ eg the average of the contribution of M to the dynamic goes 
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like ty/n so it is divergent for large n but it goes in any case to zero due to the multiplication with 
the exponential e~ Kt . Thus we expect in this case to have large coherent motions which are then 
dumped by the exponential factor. If cj = ce the situation is completely different because the 
term of the order t^/n is multiplied by a constant equal to zero and disappears. In this case the 
contribution of M converges to a gaussian random variable with zero mean and variance of the 
type of a constant +Jo(V&*t) and so we get the usual stability result. These are the meaning of 
the theorems demonstrated in the paper. 



2 Notations and formulations of the results 

Consider a dynamical system with random interactions (so-called a complex system in [9]) defined 
by 

x' = -kx + J'x, (2.1) 

where x G R n , k is a real number and J' is an n x n real random matrix. Following Rajan and 
Abbott (see QlOj)) we consider the case, when Jjj are independent Gaussian variables with mean 
values 

E\J' \ = a-{ ^/ nl/2 ' 3 = 1 >~- [M ( 2 2) 

lj \ /ie/n 1 / 2 , otherwise. 

Here < / < 1 and a > are fixed parameters, and [ii and he are chosen so that the vector 
m = (mi, . . . ,m n ) 

_ ._ / ^i/n l/ \ j = l,...[fn) 

m i-\ .. /„l/2 „^ I. „_..„-„„ \ 1 - 6 ) 



He/™ 1 ^ 2 , otherwise, 
satisfies conditions 

(m,u)=0, (m,m) = l (2.4) 

with 

u = (l,...,l). (2.5) 

The variances of Jij are chosen as follows 

E Mj - E { j; i} f } = „-V, = { (2.6) 

It is easy to see that in this case the matrix J' could be represented in the form 

J' = J + oM, (2.7) 
where the matrix M is a rang one matrix all rows equal to m, so that 

Mx=(m,x)u, x£l", (2.8) 

and the matrix J has the form 

J = n _1 / 2 W (2.9) 
where W a Gaussian matrix with independent entries satisfying conditions 

E{W ij } = 0, E{Wf 3 } = a 3 (2.10) 

with o~j defined in (|2,6p . 

As it was shown numerically in the paper [ID], the matrix J' under conditions (|2.2|) - (|2.6|) has 
a spectrum which is not localized in some fixed domain of C and so it is difficult to expect that 
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the dynamics of the system (|2.ip will be stable. But if we introduce the additional equilibrium 
conditions 

n 

Ju = ^J2 w ij = ° (* = 1» •••>»*)» ( 2 - n ) 
i=i 

then the spectrum J' coincides with the spectrum of J, which is well localized according to the 
results of [?, ?]. 

It is easy to see, that under conditions (|2.4p . f|2.8j) and (|2.1ip 

M 2 = 0, JM = (2.12) 

so that for any k > 1 

(J + aM) fc =J* + oMJ* -1 . 

Hence 

e t(J+M) = e a + a r^Me-J (2.13) 

JO 

and the solution of the system (|2.ip could be represented in the form 

Xf (t) = e - K *( e ' J x(0))i + ae- Rt f cis(e sJ x(0), m), (2.14) 

./o 

where x(0) is a vector of initial conditions. Thus to study the dynamics (I2.ip it suffices to study 
the dynamics of the system 

x' = Jx (2.15) 

with a matrix J of the form (|2.9|) . and W, satisfying conditions (|2.10p and (|2. 1 1 j) . 
Supply the system with the initial conditions 

Xi(0) = Ci + Zi, (2.16) 

where {^} are independent random variables with distributions {fi}, satisfying the conditions 

£fe} = 0, Em 2 } = af\ Em s }<C. (2.17) 

and 

cj, i = l,...[/n], ,s = f i = l,...[/n], ^(o) = f crf\ i = l,...[fn], 

c E , otherwise, l{ \ u E (x), otherwise, i \ a$ , otherwise. 

(2-18) 

Define the normalized counting function of Xi , solutions of the system ()2. 15j) , 

n 

M n {\, t) = n-^{ Xi (t) < A} = n- 1 0{X - x^t)), (2.19) 

i=l 

where 9(x) is the standard Heaviside function. jV n (A, t) is a random measure on the real line which 
counts the fraction of the variables x\, . . . ,x n which are less then A at time t. Thus it characterizes 
the distribution of Xi(t) on the real line. 

Theorem 1 Consider the system 12.15\) with a matrix J of the form \2. 9\) under conditions \2. 10\) 
and A2.11\) , and supply this system by the initial conditions A2.16\) - [2.18\) . Then for any t > 0, 

lim 7V n (A, t) = /A// (A, t) + (1 - We(X, t) (2.20) 
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where Mj(X,t) and jYe(A,£) are the convolutions of the initial distribution vi and ve with normal 
distributions AT (ci, a (t)) and N (eg , <? (t) respectively 

Afi{\,t) = (yi*M(ci,a(t)))W> M E (\,t) = (v E *N(c E ,a(t)))(\), (2.21) 
and the variance a(t) has the form 

a(t) = Aa- 1 y ^f— . (2.22) 

where 



mimi 

m=l 



0-* = fcri + (1 - f)a E 

A = (T-V /f 7 B /(l - f){ Cl - c E ? + {cnaf'f + a E af(l - /)). 



-1 „ ru ,2 . . (0), , „ J0) n f u ( 2 - 23 ) 



Theorem 2 Consider the system 112.15]) with matrix J of the form 112. 9\) under conditions 112. 1(J\) 
and 112.11}) . and supply this system by the initial conditions 112.16}) with {2.17^. Set 



w n (t)= [ ds(e sJ x(0),m). (2.24) 
Jo 

If ci / c E , then 

E{w n (t)} = n^tifatn + (1 - f)c El i E ) ~ yfii. (2-25) 

If c\ = ce, then w n (t) for each fixed t converges in distribution to a Gaussian random variable with 
zero mean and variance 



OO '111, j. 2772. 

^ = a - /yf + /4 0) + at, 1 E ^fr> ( 2 - 26 ) 

' m\m\ 

m=l 



where A and are defined in fiK 



3 Proofs 

First of all we need to compute the expectations of E{Wij} and E{WijW k i} under conditions (|2.11|) 

Lemma 1 (i) Under conditions A2.ll]) and \2. 1 0\) 

E{Wij} = 0, EiWijWu} = 5 lk (Sjiaj - ^) , (3.1) 

y rio~ * j 

where 

°* = n_1 ^2 a k = n ~ 1 ([f n \ (J i + {n- [fn})o E ). (3.2) 
(ii) Consider the random variable of the form 



z = 

k=l 



X>~ V2 ^ifc4, (3.3) 



where the coefficients d k do not depend on {W\ k }. Then z is a normal variable with zero mean and 
the variance 

<J z = n~ i ydlo- k -o-~ l {<nr i yd k o- k f (3.4) 
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Proof of Lemma [JJ The first equality in f|3. 1 1 is evident, because conditions (|2.11|) are symmetric 
with respect to the change Wit — > — Wijfc. Besides, since different lines of the matrix W have 
independent entries it is evident that for i ^ k E{WijW k i} = 0, and for i = k E{W k jW k i} do not 
depend on k. Hence, 

p/w J WjjW^ exp{- g I^ 2 /2a, - (g Wjj) 2 /(2ne)}rfW 

E{W kj W k i} = EiWxjWu) = hm = — — 

e^o J exp{- £ W{ j /2a j - W r i 3 -) 2 /(2ne)}dW 

/ dW^-Wu exp{- ^ Wg./2oj- + it J2 n~ 1/2 Wy - et 2 /2}dWdt 
~ So J exp{- ^ W^./ + it £ n-VaWy - ei 2 /2}dWdt ~ ^ 

J" (^i^i ~~ n~H 2 aj(Ti) exp{-t 2 ^ <jj/(2n)}dt 
= Jexp{-t 2 ^c7 i /(2n)}(it = 



rid* 



where dW = JJdWjy. 

3=1 

To prove the assertion (ii) of Lemma Q] we compute by the same way the characteristic function 
of z 



E{e isz } = lim 



/ exp{- £ Wfj/2^ + is Zk=i n-^Wxkdu - (£ W^) 2 /(2ne)}dW 



e -o Jexpi-^W 2 ^ - (ZW lj ) 2 /(2ne)}dW 

/ exp{- ^ W£-/2oj- + it £ n-V2Wy - et 2 /2}dWdt ^ ' ^ 



_ /exp{-X;(t + s4) 2 gj/(2n)}dt _^ /2 
/exp{-t 2 X>j/(2n)}dt 6 

Lemma [1] is proved. 

Below it will be convenient to consider the matrix J in the new orthonormal basis. Denote 

Ex =Lin{ei,...,e [/n] }, £ 2 = Lin{e [/n]+1 , . . . , e n }, (3.7) 

where {ei, . . . , e n } is the basis in which we consider the system (|2.15p initially, so that X{ = (x, e,). 

Then define in E\ and E2 the orthonormal systems {U3, . . . , U[j n ] +1 } and {u[y n ] +2 , . . . , u n } 
which are orthogonal to the vectors ei H h er^i , and e^ n ] +1 + • • • + e n respectively. If we denote 

ui = n -1 / 2 !!, u 2 = m, (3-8) 

then, according to (|2.5p . (|2.4p and our choice of 113, . . . , u n , the system {uj}™ =1 forms an orthonormal 
basis in W 1 . Let (uu, ■ ■ ■ , u n i) be the components of the vector Uj in the basis {ei, . . . , e n }. Then 
the matrix 

U = {u ki )l i=l (3.9) 

is a matrix of the orthogonal transformation from the basis {ei, . . . , e n } to the basis {ui, . . . , u n }. 
Consider the matrix J in this basis. 

J = U*JU = n~ 1/2 U*WU = nT 1/2 W. (3.10) 
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Lemma 2 The entries {Wki} k '» = i ar e independent Gaussian variables with zero means and their 
variances are 

( 0, j = 1, 

07, i = 3, . . . [/n] + 1, 



E{Wf 3 } = < 

, o-i<T E /cr*, j = 2. 



■ , , ,, (3-11) 

<?e, j = [fn\+2...,n, 



Remark 1 It follows from Lemma that the matrix J can be represented in the form 

where Ji is a matrix with i.i.d. entries Gaussian entries with zero means and variances 1, and 
D^ 1 ) is a diagonal matrix with = 0, = {oioeI o~*) X ^ 2 ^ti = °i f or i = 3, . . . [/n] + 1 

and D^ 1 = a]/ 2 /or i = [/n] + 2 . . . , n. Hence 

||J|| 2 < maxjcj/, cj£;}||Ji|| 2 . 

Using the result of ft], according to which under condition matrix 3\ with i.i.d. 

Pro6{||Jill 2 > 4 + e} < e - Cin£ \ 
and the fact that \\J\\ = \\J\\, we obtain now that 

Prob{ 1 1 J| | > 2L + e} < e~ Cn£ \ (3.12) 

where we denote 

L = maxjo-]^ 2 , c]/ 2 }. (3.13) 



Remark 2 Inequality k'3. lfy) allows us to use ||J|| in our considerations like a bounded random 
variables. Indeed, since, e.g., \x\ (t)\ < ne' l|J|l ; denoting P n (X) = Prob{||J|| > 2L + A} and using 
VS.lty , we can write for any fixed t and m,s « nj log n 

E{\ Xl {t)\ m e s ^} < 

< e s{2L+e) E{\xi{t)\ m e{2L + e - ||J||)} + n m E{e is+mt)m 6(\\ J|| -2L- 2e)} < 
< e s( - 2L+e ^E{\xi(t)\ m } + n m [ e {s+mt)x dP n {\) < e s{ - 2L+e) E{\ Xl {t)\ m } + 0{e~ Cn£ ' 2/2 ). 

J\>€ 

Hence, below we use ||J|| as a bounded variable without additional explanations. 

Proof of Lemma [2j It is evident that {Wfci}?; =1 have joint Gaussian distribution, so to prove 
Lemma [2] it is enough to compute their covariances. To this aim we use relation 

Wij = ^UkiWiauij (3.14) 
k,l 

and Lemma[TJ Then from the first equality of 13. II we derive that the mean values of {W&i}? i=1 are 
equal to zero. 

u kiii u hji u k,2i2 u hj2 

E{W klh W k2h }. (3.15) 
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Now we use the fact that for different k\ and k 2 Wk 2 i 2 are independent and for k\ = k 2 

E{Wk 1 i 1 Wk 1 i 2 } does not depend on k\ (see p.ip ). Substituting I3.1j) in (|3.15p . summing with 
respect to k\ and using the orthogonality of and u i2 , we get 

E{W hh Wi 2j2 } = Ki2^Z u hhUi 2j2 {5 h i 2 a h - o h oi 2 /{o*n)). (3.16) 

Now if ji = 1, then ui 1 j 1 = n -1 / 2 and summation with respect to 1% gives us zero because of 
(|3.2p . If ji = 3, . . . , [fn] + 1, then, since ui 1 j 1 = for l\ > [fn] + 1, we have that in the r.h.s. of 
(|3.16|) o\ x = aj, and so, using the orthogonality of Uj 1 and Uj 2 , we get the second line of (|3.11|) . If 
ji = [fn] + 2, . . . , n the proof is the same. Now we are left to prove the last line of (|3,lip . Using 
(|3.16p and (|2.4p . which gives us 



fiI = - y /(n-[fn])/[fn], » E = V[H/(n - [fn]), (3.17) 

we obtain 

[fn] n /[fn] 

E{W i2 Wi2} = n" 1 ^2 ai/ij + n^ 1 ^ (?eHe ~ (o"*" 2 ) -1 E <?Wi + E ve^e 

h=l h=[fn]+l \h=l [fn]+l 



Ml-[/n]/n) + ^[/n]/n 



(3.18) 

{ai-o E ?[fn}(n-[fn}) 



cr*n 2 



Proof of Theorem Q] Let us consider the system (|2.15p from the second equation to the last 
one as a system of equations for ^(l), . . . ,x n (t), where xi(t) is a known function. Then 



Xi(t) = x < i 1) +J2j*d S (e ( - t - s ^ m ) ij ^x 1 ( S ), (3.19) 



3=2 

where 



x\ 1] = (e ij(1) x(0)) 4 (3.20) 

and is the matrix which we obtain from J replacing the first line and the first column by zeros. 
Substituting this expressions in the first equation of (|2.15p . we get 

= £ + jf dsf nHt ~ S)xx(8) + ^i(t), (3.21) 



where 



Hence 



with 



f«(i) = n- 1 E (^^JijWiiW,-!. (3.22) 
x 1 (t) = x 1 (0) + E^|4 1) (t) + J t dsrW(t- S )x 1 ( S ), (3.23) 
4 1) (t)= [ dsxf\s), r«(t) = drr^(r) + n-y 2 Wn. 



8 



Using Lemma [H it is easy to see that 

E{{^\t)f}<C{t)n-\ (3.24) 

Indeed, according to (|3.22p . 



#> = rr^Wufu fi = ^ 1/2 E( e * J(1) )«^i 

i=2 j=2 

with fi independent of {Wy}. Hence, if we take the eighth power of (|3.22p and take the expectation 
with respect to {Wij}, we get 

E{(r£\t)f}= (3.25) 
= 7.5.3.e\ ( n- 2 ^(e* mT Be" (1) ) jlhWjll W j2l - - n^(£ ^ W ) lin W nl ) 2 J 1 < 

[ \ iiJ2 h,ji ) ) 

2- 



31 



< c(t) n - 4 , 

where J^ T means the transposed matrix of J^ T , D is a diagonal matrix such that 

Dij = 5ij(Ti, (3.26) 
and here and below we denote by C(t) function of t (different in different formulas), such that 

C(t) < Ce ct 

with some n-independent C and c. 

The relation (|3.25j) and a trivial crude bound 

\xx{t)\ < ||e' J x(0)|| < e' l|J|l ||x(0)|| < C(t)y/n 

allow us to obtain 

E{x\{t)} < (3.27) 
< 27 (E{4(0)} + 3L 2 (n- 1 ^(^(t)) 2 ) 2 +t 3 jf* dsE^{{r^{t - s)f}E^{4(s)}) < 

< 27 ((J + 3L 2 J E{(n- 1 ||e* J<1) x(0)|| 2 )} + C{t)n~ 4 J* ds^ 1/2 {sf(s)e 4s||J|l ||a;(0)|| 4 }^ < 

< C(t) (l + n- 2 J dsE 1 l 2 {x\(s)} S j , 
where L is defined by f)3. 13|) . Then, by a standard argument, we get 

E{x\{t)} < C(t). (3.28) 

This bound allows us to write (13.23D as 

xi (t) = xi (o) + x: + ( 3 - 29 ) 
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where 

E{(e^(t)) 2 }<C(t)n-\ (3.30) 

Now we can apply Lemma [H which gives us that the sum in the r.h.s. of (|3,29p is a normal random 
variable with the variance 

~4\t) = n-^Mf\t)f -^^Y,^?- (3-31) 

Define 

n 

R n (t,s) = n~ 1 '^2(TjXj(t)xj(s). (3.32) 

3=1 

Lemma 3 Under conditions of Theorem{l\ R n (t, s) 

D n {t, s) = E{(R n (t, s) - E{R n {t, s)}) 2 } < C{t)C{s)n-\ (3.33) 

^{(^'XJ^^^-^I^'XJ^W}) ^< C ( t ) n ~ 1 - (3-34) 

Besides, 

n 

E{\R n (t,s)-n- 1 ^2a j xf ) (t)xf\s)\ 2 } < C{t)C{s) n -\ (3.35) 
3=1 

Denote t t 

a n (t)= I I dt'ds'R n (t',s')-o-~\[ rf^Tx^t')) 2 . (3.36) 
Jo Jo Jo 

Remark 3 Lemma\^ implies 

E{(a^(t)-a n (t)) 2 }<C(t)n-\ (3.37) 

E{(a n (t) - E{a n (t)}) 2 } < C{t)rT l . (3.38) 
Proof of Lemma [3j To prove f)3.35|) we first estimate 

n 

« _1 E ff i^W " x fHt))Ms) - xf\s)) = (3.39) 
i=i 

n- 1 V ! S ds x f d S2 (e(*-^) j(1)T D e (^) jCl) ) il , 2 ^l^x 1 ( Sl )x 1 ( S2 )< 

' r-t \ 1/2 / r- s \ 1/2 n 



Jl J2=2 ' 

<n-W s +')ll J(1) H 
Now, using the Schwartz inequality we get 

2 



i=2 



i? n (i, a)-*" 1 (^(s) = n" 1 J>,- ((*,(*) - xf ) (t))x i ( S ) + xf(t){x (s) - xf\s))) 



3=1 



<Cn-h S e 2 ^ 3W W (j\ 2 ( Sl )d Sl + J\ 2 (s 2 )ds^J | |x(0)| | 2 £ W£. (3.40) 
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Combining this with (|3.28p we obtain (|3.35p . To prove (|3.33|) . we write 



D n (t,s) = J2 WjE^ (xi(t)xi(s) - E{xi(t)xi(s)}^ (xj(t)xj(s) - E{xj(t)xj(s)}j J. (3.41) 



Let us estimate, e.g. E<. f x\{t)xi{s) — E{xi(t)x\(s)} ) ( x 2 (t)x 2 (s) — ^{^2(^)^2(5)} ) f- To this 
end we write the representations ( cf. (|3.23ft ) 



x x (t) = xi (0) + n^Wuh (t) + n-V2 ^ W y d< 1,2) (t) 

+ / dt'r^ 1 \t-t')x 1 (t') + [ dt'r^ 2 \t-t')x 2 (t'), 



(3.42) 

X2 (t) = x 2 (0)+n- 1 / 2 W 21 d 1 (t)+n- 1 / 2 J2 w 2jdf ,2) (t) 

t t j=3 
+ f dt'rV' 1 \t-t')x 1 (t')+ [ dt'r^ 2 \t-t')x 2 (t'), 
Jo Jo 

where 

df 2 \t) = j\sx^\s), x?*\t) = {e"™xi0)) jt rfrt»{t) = [drr^\r), 

n 

r^\t)=n-^(e t3(ia) hW m W jP . 

i,j=2 

j( 1,2 > is the matrix J with the first and the second lines and the first and the second columns 
replaced by zeros. Similarly to (|3.24p - (|3.30j) we obtain 

n 

x x (t) = Xl (Q) +n -V 2 Y,W l3 df 2 \t) +e^(t), 

3=3 

x 2 (t) = x 2 (0)+n-^J2w 2j df' 2 \t) + e^(t), (3 ' 43) 

i=3 

E{(e£\t))} < n^Cit), E{(r (2 \t))} < n - l C{t). 

Then we can write 
£ j (xi(t) Xl (s) - E{xi(t)xi(s)}j (x 2 (t)x 2 (s) - E{x 2 (t)x 2 (s)} 

< f I" I f S dtids^dtzds^ElfR^^t^s^-EiR^^t^si)} 
Jo Jo Jo Jo IV 

(V' 2 )(44) - E{R^(t' 2 , S ' 2 )}) ) + C(t)C{s)n- 1 

ft i-s 1 

< 



II [ II [ dt ' ids ' idt ' 2ds ' 2 W( i?(i ' 2) ^' s/ i)-^ i?(i,2) ^' s 'i)> 

+ e{ (V 2 V 2 , 4) - e{r {1 ' 2) (4 4)}) 2 } 



C(t)C(s)n 



< 



2ts J* J* dt'ds'E^ (rW^, s') - E{R (1 ' 2 \t', s')}j I + C(t)C{s)n-\ (3.44) 
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where 



Similarly to fj3.35j) 



i=3 



E{(R^ (t, s) - Rn(t, s)) 2 } < C{t)C{s)n-\ 
Repeating (|3,44p for all terms in (|3.4ip with different we obtain the inequality 



D n (t,s) <2a 2 ts f f dt'ds'D n ' J ' 
Jo Jo 



Iterating this inequality M = [logn] times, we get (|3.33|) . The proof of the inequality (|3.34[) follows 
from the representation (|3.42ft immediately, if we use the independence of x\(0) and X2(0). 
Lemma [3] is proved. 

Now we are ready to prove the self averaging property of A/" n (A, t), as n — > oo, i.e. we prove 
that for any real A and t > 



lim E\[M n (X,t)-E{M n (X,t)}) \ = 0. 



(3.45) 



According to the standard theory of measure, for this aim it is enough to prove that g n (z,t) - the 
Stieltjes transform of the distribution A/" n (A, t) 



g n (z,t) 



dM n {X,t) 
X-z 



n 



n 



(346) 



for any z : Qz / possesses a self averaging, property. 
Lemma 4 For any z : Qz ^ 



lim E 

n—+oo 



g n (z,t) - E{g n (z,t)} 



(3.47) 



Proof of Lemma [4j Similarly to (|3.4ip we write 

2~ 



E 



g n (z,t) - E{g n (z,t)} 



n 



] J2 W (*<(*) 



z)-\xj(t) - z)- 1 \ - El (xi(t) - z)- 1 \e{ (xj(t) - z) 



(3.48) 



Then repeating the arguments (|3.4ip - (|3.43p . we obtain 

2- 



E 



g n (z,t) - E{g n (z,t)} 



< C{t)n~ l + n~ 2 E { ( F i(°n, z) - E{Fi(a n , z)}^j (f^, z) - E{F 3 {a n , z)}\ \, (3.49) 



where 5 n is defined by (|3.36p and we denote 



1 



dvi{x)dye y2 / 2 



V2vr J J x + Ci + a 1 ' z y - z 
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Since evidently 



we get from (|3,49p 



I 1/2 1/2, 

\F i (a 1 ,z)-F i (a 2 ,z)\<C 1 -^- ^ 



E 



g n (z,t) - E{g n (z,t)} 



2~ 

' <C{t)rT l + C\%z\- 2 E{(a n -E{a n }) 2 }. (3.50) 



Now the assertion of Lemma |4] follows from (13.381) . 

Using (|3.35p . the s.a. properties (I3,33p . and the fact that the system fl2. 15f) is symmetric with 
respect to x±, . . . , X[j n ] and with respect X[y n ] +1 , . . . , x n , we obtain 

E{a n (t)} = ?^E ^J* dsx^sfj ^+a E (l-[fn]/n)E^J*dsx n (s) S J |- 

a- 1 (^^^daE{x 1 (a)} + a E 0--\fn]/n)J t d8E{x 1 (8)A +o(l). (3.51) 
Repeating our conclusions for x n (t), we get 

x n (t) = x n (0) + E^7f4 n) (*) + £ - n) ( t )' ( 3 ' 52 ) 

where 



E{(e^(t)) 2 }<C(t)n-\ (3.53) 
4 n) (*)= f4 n \s)ds (i = l,...,n-l), 



Jo 

x^ = (e* J<n) x(0))j and the matrix is obtained from J by replacing the last line and the last 
column by zeros. Then, applying Lemma [H we obtain that the second sum in (|3.52|) is a Gaussian 
random variable with the same variance a n {t) (see (13.5ip ). 
Equations (|3.3ip and (I3,52p combined with (I2.16P give us 

E{ Xl (t)} = a, E{x n (t)} = c E . (3.54) 

Denoting 

R^{t,s) = £?{xi(t)xi(s)}, R®(t,s) = E{x n (t)x n {s)}, (3.55) 
we obtain from (|3.3ip and (|3.52p the system of equations 

R$t\t,8) = E{xl(0)} + aif I f R^{t',s')dt'ds' 

t s 

+a E {l-f) I f R^(t',s')dt'ds' -tsa^iajfd + UEil- f)c 2 ) 2 + o(l), 
Jo Jo s 

R^\t,s) = E{x 2 n (0)} + aif f [ R^(t',s')dt'ds' 

t 

+a E {l-f) [ I R^(t',s')dt'ds' -tsa-^ajfa + UEil- f)c 2 ) 2 + o(l). 
Jo Jo 

(3.56) 

Then we obtain that the function 

4°>(i,s) = ajfR^(t,s)+a E (l - f)R%\t, s) - a? Ma + a E (l - f)c 2 ) 2 , (3.57) 
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satisfies the equation 

R ( £ ) (t,s) = A + a* f f R^(t',s')dt'ds' + o(l), (3.58) 
Jo Jo 

where 

A = a!fE{x 2 M} + a E (l - f)E{x 2 n (0)} - a^aifd + a E (l - /)c 2 ) 2 = (3.59) 
= <^/(l -/)(<*-<*)* + (aja ( 0)/ + _ /)} . 

As it was proved in [7] this equation has the unique solution 

( °° mxm m \ 
1 + E^4n- +°(!)- ( 3 - 6 °) 
^-^ m\m\ j 
m=l / 

Then we can easily find that 

00 'tTl J.771 771 

R%\t,8) = E{xl(0)} + Aa- 1 J2^T I r + o ^ 

^-^ m!m! 

m=l 

< ^~ > TTL 4-TTL TYl 

R%\t,s) = £{4(0)} + Ax- 1 £^ T ^- + o(l) 

m\m\ 

m=l 

Hence 



(3.61) 



/*i /*i TTb-L^TTl 

a(t) = lim / / dt'ds'R^(t',s') = Aa^ 1 V ^f— . (3.62) 
™-Wo Jo " ™ ! ™ 

Using (|3.45p . and the symmetry of the problem we obtain that M n (t, A) converges in probability to 

fE{9(\ - Xl (t))} + (1 - f)E{9{\ - x n [t))}. 
But by the above arguments 

E{6{\ - x x (t))} - f_^=j dvi{x) exp{(* - a x /\t)y - cj) 2 /2}, 
E{9(X-x n (t))}^ [ X I ' du E (x)^ V {{x-d 1 l 2 {t)y-c E ) 2 /2}. 

J-oo V27T J 

Theorem 1 follows. 

Proof of Theorem [2j To prove Theorem [2] it is convenient to consider the system (|2. 15|) in 
the basis {uj}™ =1 defined above (see (|3.7p - (|3.9p . Let 

Vi(t) = (x(t),ui) (3.63) 

Then the system ()2. 15j) takes the form 

y = Jy, (3.64) 

where J is defined by (|3.10p . The question of interest is the behavior of y 2 (£)- Repeating for y 2 (i) 
the arguments (|3.19p - (13.24p we get the representation 

y 2 (i) = y 2 (0) + J 22 f t y 2 (s)ds + / dsrjpQt - s)y 2 (s), (3.65) 

Jo n L/2 Jo 
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with 

[/«] (0) n (0) 

w(0) = (a(0),m) = Vn(cjMj/ + CBMl-/)) + /*/y)^7= + ^ ]C ^> ( 3 - 66 ) 

i=l V i=[/n]+l V 

dj(*) = fdsyf{s), = (e t5(2) y(0)),, 

r ( 2 )(t) = / f*0( T)dr + n -% ffll f«(t) = n- 1 V (e^^A 

where and j( 2 ) is the matrix which we obtain from J replacing the first and the second lines and the 
first and the second columns by zeros. Taking the expectation in (|3.65j) we get the first statement 
of Theorem [2j Now assume that cj = ce = c. Then, repeating arguments (|3.22p - (|3.28p . we get 
that 

n rjr 

m(t) = 2/2(0) + ^djit) + i n (t), (3.67) 

5=3 

and 

E{(eW(t)) 2 }<C(t)n~\ (3.68) 

Applying Central Limit Theorem to the r.h.s. of (j3.66j) it is easy to obtain that 2/2(0) converges in 
distribution to a Gaussian random variable with zero mean and the variance 

<?y = ftfvi + (1 - f)VE°E = (1 - /)c>7 + fcTE- 

Besides, since {Wjjj} are independent Gaussian random variables, the sum in the r.h.s. of ([3.65ft 
is a gaussian random variable with the variance 

LM+l n 
= n -V, Y, (df\t)f + n-i*E £ (df\t)f = 

3=3 j=[fn]+2 

ifn]+l ,. t , t n ,. t 

= n~ 1 a I dt'ds'yj(t')yj(s) +n~ 1 a E dt ' dsy j (t')y j (s) + o(l) = 

3=3 Jo Jo j=[M+2 h Jo 

(3.69) 

«t ft n [/n]+l t t 



n 



~Vj / / dt'ds' ^2 x k(t')x k '(s') ^2 UkjUvj+TT^E / / dt'ds' £ x k (t')x k >(s') £ u k jU k >j. 
Jg ^° fc,fc'=l i=3 ^° ^° fc,fe'=l i=[/n]+2 



J=[/n]- 

If 1 < k < [fn], while [fn] + 1 < k' < n or 1 < k' < [fn], while [fn] + 1 < k < n, then by 
construction of U3, . . . , u n 

IM+l n 

UkjU k 'j = 0, £ u kj u k 'j = 0. 

i=3 j=[/n]+2 

Let 1 < A;, A;' < [/n]. Then 

n 

y~] u kj u k >j = 

3=[fn]+2 

and so 

u k ju k 'j = ^UkjUk'j = 5 kjk ' - UkiUk'i - u k2 u k i 2 = Sk,k' - n~ l (l + uj). 

3=3 j=3 
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Similarly for [/re] + 1 < k, k! < n, we get 

n 

^ UkjUk'j = Sk,k' - n _1 (l + He)- 
j=LH+2 

Finally we get 

= (3.70) 
= n~ 1 ai / / dt'ds' V x k (t')x k ,(s')(5 ktk > - + 



+n~ 1 aE / dt'ds' ^ Xk(t')x k '(s')(Sk,k' - n _1 (l + = 
^° fe,fc'=[/n]+l 

M T [ dt'ds' RW(t',s')+(l-f)<r E I I dt'ds' R^{t' , S ')-t 2 (a / / 2 c 2 (l+/i?)+a s (l-/) 2 c 2 (l+ / u| 
= / / dt'ds' R^\t\s') =Ao- 1 V +o(l). 



m\m\ 

m=l 

Since the sum of independent Gaussian variables is a Gaussian random variable with the variance 
equal to the sum of variances, we obtain that w n {t) converge in distribution to a Gaussian random 
variable with zero mean and the variance 

aW=aM(t) + a y . 

The second assertion of Theorem [2] follows. 



References 

[1] JE Cohen, and CM Newman. The stability of large random matrices and their products. Ann. 
Prob. 12, 283-310, 1984. 

[2] YC Deng, MZ Ding, and JF Feng. Synchronization in stochastic coupled systems, Jour. Phys. 
A. 37, 2163-2173, 2004. 

[3] IJ Farkas, I Derenyi, AL Barabasi, and T Vicsek. Spectra of "real- world" graphs: beyond the 
semicircle law, Phys. Rev. E. 64, 026740, 2001. 

[4] JF Feng, and B Tirozzi. The SLLN for the free-energy of a class of neural networks Helv. Phys. 
Acta 68, 365-379, 1995. 

[5] JF Feng, V Jirsa, MZ Ding (2006) Synchronization in Networks with Random Interactions: 
Theory and Applications Chaos (invited review). 

[6] JF Feng, M. Shcherbina, and B. Tirozzi. On the critical capacity of the Hopfield model, 
Commu. Math. Phys. 216, 139-177, 2001. 

[7] JF Feng, M. Shcherbina, and B. Tirozzi. Dynamical behavior of a large complex system, 
submitted to Markov Processes and Related Field. 

[8] A Guionnet, and O Zeitouni. Concentration of the spectral measure for large matrices Elec- 
tronic J. Prob. 5, 119-136, 2000. 



16 



[9] R. May. Will a large complex system be stable? Nature 238, 413-414, 1972. 

[10] K. Rajan , L.F. Abbott, Eigenvalue Spectra of Random matrices for Neural Networks, Phys. 
Rev. Lett. 97, 188104, 2006 

[11] L Pastur. Random Matrices as Paradigm In: Mathematical Physics 2000, 216-266, 
A.Grigoryan, T. Kibble, B. Zegarlinskii (Eds.), World Scientific, Singapore, 2000. 

[12] L Pastur, and M Shcherbina. The absence of the selfaverageness of the order parameter in the 
Sherrington-Kirkpatrick model. J. Stat. Phys. 62, 1-19, 1991. 

[13] A Soshnikov. A note on universality of the distribution of the largest eigenvalue in certain 
sample covariance matrices J. Statistical Physics 108, 1033-1056, 2002. 

[14] M Shcherbina, and B Tirozzi. The free energy of the class of Hopfield models. Jour. Stat. Phys. 
72, 113-125, 1993. 

[15] E Wigner. On the distribution of the roots of certain symmetric matrices, Ann. of. Math. 67, 
325-327, 1958 



17 



